VDR Follow Up – Tonemapping for HDR Signals

This is the third post in the follow up series to my prior GDC talk on Variable Dynamic Range. Prior posts covered dithering, today’s topic is a collection of practical advice on adapting a game’s tonemapping pipeline to both traditional display signals and new HDR output signals.

Variable Peak Brightness

We can source baseline HDR TV properties directly from the Ultra HD Premium certification requirements: OLED displays must reach more than 540 nits, and LCD displays must reach more than 1000 nits. TVs prior to supporting HDR inputs often exceeded the OLED 540 nit baseline. As of this post HDR LCD TVs tend to exceed 1000 nits a little but not by a full stop, and can even vary temporally. The certification requirements do not include any color calibration requirements nor do they include any temporal requirements on sustaining peak brightness. Reaching peak brightness works against practical consumer power and possibly thermal constraints. For example the edge-lit Samsung UE55KS8000 HDR TV as reviewed by HDTVTest reached 1420 nits prior to calibration “needing at least 7 seconds to ramp up from circa-700 nits to 1400 nits. The display would then hang onto 1400 nits for a further 13 seconds, before peak luminance slowly dropped to 520 nits” (quoting from the review).

Typical consumer HDR TVs can range from 0 to a little over 1 stop brighter than high quality pre-HDR TVs. Note that ambient room lighting can range roughly 15 stops from night (with no lights on) to day time (with unblocked windows). To place this in context in terms of perceived brightness, an older non-HDR display at night can appear many times brighter than an HDR display in the day. So HDR and non-HDR displays both share the exact same content authoring challenge, users will experience the game in both low contrast (aka LDR) and high contrast (aka HDR) viewing conditions on both kinds of displays.

While HDR signals are absolute signals from 0 to 10000 nits, the Ultra HD Premium certification does not include any fixed tonemapper algorithm. It is up to the TV to decide how to map the 0 to 10000 nit input range into the limited output capacity of the HDR TV. The TV’s tonemapper choice can have a variable effect on the perceived brightness of a video signal. Some TVs can bias the signal darker during tonemapping to increase the contrast between peak brights and midtones.

The above factors point to the requirement of having an easily accessible in-game exposure control .

Ideally the user would be able to adjust exposure while looking at the game as paused during play (with no darkening or full-screen effects applied). Exposure range should be +/- many stops to cover the difference between night and day viewing.

HDR Signal = Two Part Tonemapping

HDR signals have an absolute scale, and HDR10 signals must fill the full 0 to 10000 nit output range. This means that games targeting HDR signals get a two part tonemapping. First the game must tonemap the larger range of render output into this 0 to 10000 nit signal range. Then the TV will map the 0 to 10000 nit range into the TV relative range. Lets break this down into some numerical examples, and disregard gamut and dithering issues for now.

Traditional Non-HDR Example : Game renders to {0 to 65504} (peak FP16 value). Game then tonemaps that to {0 to 1} which roughly represents say {0 to 500 nits} on a good non-HDR TV. Traditional non-HDR signals are display relative scale. Finally the game transforms the linear output to sRGB or Gamma 2.2 or Rec709 (depending on desired target) followed by writing to the back buffer.

HDR Example : Let’s assume the HDR output is still {0 to 1}, and this now represents {0 to 10000 nits}. This aligns with APIs which support a 32-bit 10:10:10:2 output with PQ transform already applied by the application. This requires roughly a 500 nit / 10000 nit or 1/20 scale factor. Game renders to {0 to 65504}. Then tonemaps that to {0 to 1} using output mid-level ( midOut in my Advanced Techniques and Optimization of HDR Color Pipelines GDC presentation) multiplied by this 1/20 scale factor, and finishes by applying the linear to PQ transform followed by writing to the back buffer.

So both the non-HDR and HDR paths at this point are effectively the same shader code, with the exception that the HDR path requires a more expensive PQ transform at the end (instead of the faster sRGB output for example).

Note the magnitude of this 1/20 scale factor is a little over 4 stops. That is a rather large change. This means that having an in-game tonemapper which, outside the highlight compression, maintains RGB ratios (as in my GDC presentation) is critical. Otherwise the in-game switch to/from HDR signal output will do bad things like effect saturation in the darks, etc. This will be a problem with traditional tonemappers which apply curves separately to R, G, and B channels.

The midOut output mid-level in my GDC presentation is also the in-game exposure control from the prior section. So switching in and out of HDR signal mode is simply applying an extra hidden scaling on that exposure control.

In practice this 1/20 factor won’t be exactly 1/20. The way to come up with the correct scaling factor is to have one machine with a calibrated non-HDR output serve as a “master”. Then take a second machine driving an HDR output on the same content and manually find the scale factor which best matches midtones to the “master” across a collection of HDR TVs. This is required because each TV can have a different tonemapper (with different hidden internal exposure bias which might even change based on TV user controls). The result will be a good initial estimate. And in practice the individual user will fine tune using the in-game exposure control .

Managing the HDR and Non-HDR Mode Switch

When windowed, the game outputs a non-HDR signal. When the game transitions to exclusive full-screen and enables HDR, the display driver is going to do a hidden mode switch during the v-blank interval, and then start introducing HDR meta-data into the non-visible part of the display signal. The TV sees the HDR meta-data and responds by interpreting the signal as an HDR signal.

Assuming the TV has the same peak brightness in both the HDR and non-HDR modes, assuming the desktop when windowed is not bright (which would introduce global dimming), and assuming the 1/20 scale factor calibration works for the TV, the output in both the HDR and non-HDR mode should look similar (at least in the midtones), and both will look like they have “HDR”. The differences will be as follows,

• Changes in Gamut.
• HDR mode will affect brights and highlights differently than the non-HDR mode. In HDR mode the game will look different depending on TV tonemapper. Also techniques from my GDC talk, like maintaining saturation in highlight compression using improved crosstalk, cannot be applied in HDR mode because the TV is doing the majority of highlight compression.
• HDR mode might also affect darks depending on if the TV’s tonemapper does contrast adjustments.

One obvious user-interface challenge with the HDR mode switch is that the in-game exposure control as described is overloaded for both night to day viewing conditions, and fine tuning the non-HDR to HDR bias for the TV. A suggestion would be to split this out into two user-facing controls: “exposure” and “HDR exposure bias” (which would default to the 1/20 scale factor). Best to keep both those controls working in stops (i.e. log2 scale) so movement on the control is perceptually uniform.

I didn’t touch on Gamut issues in this post as that is large topic which deserves a separate post. Until next time . . .